Integrated circuit image alignment and stitching

ABSTRACT

In general, techniques are described for processing a set of high-resolution images of an integrated circuit, the images captured at different locations with respect to the integrated circuit, to automatically align and “stitch” the set of high-resolution images into a larger composite image. For example, an imaging system as described herein may use sampled feature points distributed across different grid tiles within overlap regions for pairs of images to match feature points to inform the alignments of a pair with respect to each image in the pair. The system may in some cases further apply a bundle adjustment to iteratively align and refine the alignment results for each image in a set of images being processed. In some examples, the bundle adjustment is a best-fit adjustment based on minimizing the net error associated with the alignment of the set of images.

RELATED APPLICATIONS

This application claims the benefit of U.S. Provisional Application No.62/821,557, filed Mar. 21, 2019, which is incorporated by referenceherein in its entirety.

GOVERNMENT LICENSE RIGHTS

This invention was made with Government support under FA8650-15-C-9111awarded by the United States Air Force. The Government has certainrights in the invention.

TECHNICAL FIELD

This disclosure generally relates to image processing and, morespecifically, to image alignment.

BACKGROUND

Imagery processing systems may store and manage imagery (e.g. imagefiles), and may enable such imagery to be viewed, edited, processed,analyzed, tagged, categorized, and the like. For example, a microscope,such as a scanning electron microscope, may have an image-capture deviceconfigured to capture high-resolution images of microscopic surfaces andtransfer the images to a processing system.

SUMMARY

In general, techniques are described for processing a set ofhigh-resolution images of an integrated circuit, the images captured atdifferent locations with respect to the integrated circuit, toautomatically align and “stitch” the set of high-resolution images intoa larger composite image. For example, an imaging system as describedherein may use sampled feature points distributed across different gridtiles within overlap regions for pairs of images to match feature pointsto inform the alignments of a pair with respect to each image in thepair. The system may in some cases further apply a bundle adjustment toiteratively align and refine the alignment results for each image in aset of images being processed. In some examples, the bundle adjustmentis a best-fit adjustment based on minimizing the net error associatedwith the alignment of the set of images and may computed by solving anoptimization problem that is weighted by the number of matched featuresamong the images.

The techniques may provide one or more technical advantages thatfacilitate one or more practical applications. For example, by samplingthe feature points across grid tiles within the overlap regions, thetechniques may facilitate sufficient distribution of unique featurepoints for comparison between pairs of images. Moreover, in some cases,the use of bundle adjustment may address and resolve large numbers ofseams that may otherwise appear in a composite image being generated byaligning a set of images.

In one example, the disclosure is directed to a method of stitchingimages to generate a composite image of an object, comprising: by acomputing system, for each pair of two or more pairs of the images:partitioning a first overlap region of a first image of the pair into afirst plurality of grid tiles and partitioning a second overlap regionof a second image of the pair into a second plurality of grid tiles, thefirst overlap region and the second overlap region being overlappingregions of the first image and the second image; extracting at least onefeature point from each of the first plurality of grid tiles and atleast one feature point from each of the second plurality of grid tiles,wherein the feature point defines a feature on an image, the featurebeing on a submicron or nanoscale level; determining correspondingfeature points between the at least one feature point of the firstoverlap region and the at least one feature point of the second overlapregion; and aligning the first image with the second image based on thecorresponding feature points to determine at least one transformationmatrix for the pair for transforming the first overlap region of thefirst image to the second overlap region of the second image; computing,by the computing system based on the respective at least onetransformation matrices for the two or more pairs, global coordinatesfor each of the images by performing a bundle adjustment for at leasttwo of the two or more pairs of the images to reduce a net errorassociated with the alignment of the images; stitching, by the computingsystem, the images based on the respective global coordinates for theimages to generate the composite image; and outputting, by the computingsystem, at least one of the composite image and the respective globalcoordinates for the images.

In another example, the disclosure is directed to a system. The systemincludes an image-capture device configured to capture images of anintegrated circuit; and a computation engine executing on one or moreprocessors and configured to, for each pair of two or more pairs of theimages: partition a first overlap region of a first image of the pairinto a first plurality of grid tiles and partition a second overlapregion of a second image of the pair into a second plurality of gridtiles, the first overlap region and the second overlap region beingoverlapping regions of the first image and the second image; extract atleast one feature point from each of the first plurality of grid tilesand at least one feature point from each of the second plurality of gridtiles, wherein the feature point defines a feature on an image, thefeature being on a submicron or nanoscale level; determine correspondingfeature points between the at least one feature point of the firstoverlap region and the at least one feature point of the second overlapregion; and align the first image with the second image based on thecorresponding feature points to determine at least one transformationmatrix for the pair for transforming the first overlap region of thefirst image to the second overlap region of the second image. Thecomputation engine is further configured to compute, based on the atleast one transformation matrices for the two or more pairs, globalcoordinates for each of the images by performing a bundle adjustment forat least two of the two or more pairs of the images to reduce a neterror associated with the alignment of the images. The computationengine is further configured to stitch the images based on therespective global coordinates for the images to generate a compositeimage. The computation engine is further configured to output at leastone of the composite image and the respective global coordinates for theimages.

In another example, the disclosure is directed to a computer readablestorage medium containing instructions. The instructions, when executedby at least one processor of a computing device, cause one or moreprocessors to execute a computation engine configured to: for each pairof two or more pairs of images of an integrated circuit: partition afirst overlap region of a first image of the pair into a first pluralityof grid tiles and partition a second overlap region of a second image ofthe pair into a second plurality of grid tiles, the first overlap regionand the second overlap region being overlapping regions of the firstimage and the second image; extract at least one feature point from eachof the first plurality of grid tiles and at least one feature point fromeach of the second plurality of grid tiles, wherein the feature pointdefines a feature on an image, the feature being on a submicron ornanoscale level; determine corresponding feature points between the atleast one feature point of the first overlap region and the at least onefeature point of the second overlap region; and align the first imagewith the second image based on the corresponding feature points todetermine at least one transformation matrix for the pair fortransforming the first overlap region of the first image to the secondoverlap region of the second image. The instructions further cause theone or more processors to execute the computation engine to: compute,based on the respective at least one transformation matrices for the twoor more pairs, global coordinates for each of the images by performing abundle adjustment for at least two of the two or more pairs of theimages to reduce a net error associated with the alignment of theimages; stitch the images based on the respective global coordinates forthe images to generate the composite image; and output at least one ofthe composite image and the respective global coordinates for theimages.

The details of one or more aspects of the disclosure are set forth inthe accompanying drawings and the description below. Other features,objects, and advantages of the disclosure will be apparent from thedescription and drawings, and from the claims.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an example system for processinghigh-resolution images of an integrated circuit.

FIG. 2 is a block diagram illustrating an example system for processinghigh-resolution images of an integrated circuit, according to techniquesof this disclosure.

FIG. 3 is a flowchart illustrating a method of processing images of anintegrated circuit, according to techniques of this disclosure.

FIG. 4 is a flowchart depicting an image-processing pipeline for imagesof an integrated circuit, according to techniques of this disclosure.

FIG. 5 is a block diagram illustrating distributed feature-matching of apair of images of an integrated circuit, according to techniques of thisdisclosure.

FIG. 6 is a flowchart illustrating a method ofdistributed-feature-matching a pair of images of an integrated circuit,according to techniques of this disclosure.

FIG. 7 depicts a bundle adjustment of pairwise registrations of imagesof an integrated circuit, according to techniques of this disclosure.

FIG. 8 depicts a frequency analysis of images of an integrated circuit,according to techniques of this disclosure.

FIG. 9 is a flowchart depicting a method of aligning images of anintegrated circuit using a Deep Neural Network (DNN), according totechniques of this disclosure.

FIG. 10 is a flowchart illustrating a method of aligning images of anintegrated circuit using frequency analysis to generate a set oftraining data for a DNN, according to techniques of this disclosure.

FIG. 11 is a flow chart depicting a method of processing images of anintegrated circuit, according to techniques of this disclosure.

Like symbols refer to like elements throughout the figures and the text.

DETAILED DESCRIPTION

Large area de-processing and nanometer scale imaging tools forintegrated circuits, or “chips,” are now available. For example, aplasma source focused ion beam scanning electron microscope (FIB-SEM)approach can delayer a large chip area by removing integrated circuitmaterials. However, defect validation in failure analysis cases andanti-tampering verification of state-of-art integrated circuits can onlybe performed in isolation with limited scope and speed in analysis. Fullscale analyses of the whole integrated circuit die is problematicbecause, as minimum device feature size shrink and the number of devicesmultiply, the complexity of the image processing and the size of theglobal integrated circuit image data base poses an ever-challengingworkload. Full scale integrated circuit analysis is important becausestate-of-art integrated circuits are particularly vulnerable toinsertion of sparse modification that are difficult to identify and cancompromise critical functionality and reliability.

The images can be low contrast, depending on the microscope settings.Techniques are described herein for improving contrasts of the images.Chip images have high resolution of several pixels per nanometer.Because circuit structures are very small (in the order of nanometers,for advanced microelectronics), each image is quite large. Consequently,the composite “mosaic” images generated by stitching together the imagesare equally large and are composed of many stitched seams. Techniquesare described herein for using adjust the alignment of those many seams.

For large chip dies, the challenges in chip delayering is theconsistency and robustness. That is, the surface quality needs to begood and uniform so that image quality does not degrade. Furthermore,the surface plane should be planar such that imaging can be consistenton a single integrated circuit layer. For example, a bevel delayeringoutput would cut across multiple integrated circuit layers includingmetal and via layers. Furthermore, with non-planar transistors in smallgeometry transistors, ultra-planar delayering may be important toproperly extract transistor circuitry from captured images.

The surface of an integrated circuit (IC), also referred to as asemiconductor chip, can be imaged to reveal the surface-level (orexterior) electrical interconnectivity of the circuit. Additionally,successive delayering of the chip can be performed to reveal theinternal circuit structure below the exterior surface, and display theunderlying gate-level netlist. The specific circuit functionality (e.g.adder, multiplier, A/D converter, etc.) may then be inferred from thegate-level netlist. Chip delayering may either be performedmechanically, such as via mechanical polishing, an ion-beam, or achemical polish, or alternatively, through optical imaging. Such imagingmay be captured, for example, with a high-power electron microscope, orwith x-ray devices. The resulting images are often high-resolution,nanometer-scale pictures with detailed features, each individual imagedepicting a different section of the chip. The features can be on asubmicron level or a nanoscale level, for instance. Accurate automaticalignment of the individual images into a single composite image of theentire chip can often be difficult, as the detailed features of theinternal structure of the chip often display repetitive, edge-basedpatterns, resulting in ambiguous alignment along the various iterationsof the patterns. Traditionally, stitching algorithms are designed fornatural scenes, and natural images are visually more forgiving. Incomparison, circuit images have edge-based content, and lesstexture/color content. Circuit images have more repetitive patterns,which makes traditional stitching algorithms fail to match correspondingpoints in overlap regions. For example, images where single directionalinterconnect wires dominate, picket-fence artifacts may make the imageregistration difficult, and in turn, produces poor quality stitches.

This disclosure further describes techniques for high-accuracy alignmentof chip images into a composite image that enables robust stitchingresults across imaging and common delayering issues. Example processingpipelines and steps for stitching together high-resolution circuitimages to form a composite mosaic image are described.

FIG. 1 is a block diagram illustrating an example system for processinghigh-resolution images of an integrated circuit, according to techniquesof this disclosure. Imaging system 100 is configured to capture a set ofhigh-resolution images of integrated circuit 102, accurately align theimages, and stitch them into a single composite “mosaic” image.

Imaging system 100 includes a magnification device fitted with animage-capture device (ICD) 108, configured to capture high-resolutionimages, for example, at a nanometer scale. ICD 108 may represent ahigh-resolution camera. In the example of FIG. 1, a magnification deviceis depicted as scanning electron microscope (SEM) 104. In otherexamples, the magnification device and ICD 108 may include an x-ray,other electromagnetic-based optical magnification and imaging device, orother imaging system for capturing multiple high-resolution images ofdifferent parts of integrated circuit 102.

SEM 104 includes microscope stage 110. Stage 110 includes a planarsurface on which integrated circuit 102 may be placed, so as to bepositioned underneath (i.e., in the field of view of) image capturedevice 108. SEM 104 may include controller 112, configured to spatiallymove microscope stage 110 with respect to ICD 108, or alternatively, tospatially move ICD 108 with respect to microscope stage 110. Forexample, controller 112 may be configured to perform a horizontaltranslation (in the x-y plane) of microscope stage 110, exposingdifferent sections of integrated circuit 102 to ICD 108. In someexamples, controller 112 may also be configured to perform a rotation(in the x-y plane) of stage 110, or configured to move stage 100vertically (in the z-axis direction) closer to or farther away from ICD108, effectively scaling the resulting images. In some examples,controller 112 and/or microscope stage 110 may be configured to output aset of microscope stage coordinates, indicating an approximate relativeposition of stage 110 with respect to ICD 108. These stage coordinatesat which an image is captured may be associated with the image.

Imaging system 100 includes computing device 106. Computing device 106is configured to receive, via a wired or wireless link, a set of images120 of integrated circuit 102 from ICD 108. Computing device 106 mayinclude memory 114 configured to store images 120. In some examples,computing device 106 and ICD 108 are integrated within a single device.

Computing device 106 includes a computation engine 116, e.g., a memoryfor storing instructions and processing circuitry for executing suchinstructions, configured to execute an image processor 118 (detailedfurther in FIG. 2) to process images 120. Image processor 118 isconfigured to analyze, align, and “stitch” images 120 into a singlecomposite image 122.

In accordance with techniques described in this disclosure, imagingsystem 100 may use sampled feature points distributed across differentgrid tiles within overlap regions for pairs of images 120 to matchfeature points to inform the alignments of a pair of images with respectto each image in the pair. As a result, the imaging system 100 mayfacilitate sufficient distribution of unique feature points forcomparison between images in pairs of images 120. Imaging system 100 mayin some cases further apply a bundle adjustment to iteratively align andrefine the alignment results for each image in a set of images 120 beingprocessed. The use of bundle adjustment by computing device 106 mayaddress and resolve large numbers of seams that may otherwise appear ina composite image being generated by aligning images 120, and bundleadjustment generates global coordinates for each of images 120. Usingthe global coordinates for each of images 120, image processor 118stitches the images 120 to generate a composite image 122. Computingdevice 106 may output composite image 122 to output device 124, which atleast in some examples may represent a display screen or storagelocation, e.g., a disk or a local or remote storage server. In this way,computing device 106 may perform high-accuracy alignment of images 120and stitching into a composite image 122, thus enabling robust stitchingresults across imaging and common delayering issues. In some cases,computing device 106 may output respective global coordinates for images120 in addition to, or alternatively, to outputting composite image 122.

FIG. 2 is a block diagram illustrating an example system, with exampledetails, for processing high-resolution images of an integrated circuit,according to techniques of this disclosure. In this example instance,imaging system 100 includes computing device 106 configured to receive,via input device(s) 140, a set of high-resolution images 120 of theintegrated circuit.

Memory 145 may store information for processing during operation ofcomputation engine 116. In some examples, memory 145 may includetemporary memories, meaning that a primary purpose of the one or morestorage devices is not long-term storage. Memory 145 may be configuredfor short-term storage of information as volatile memory and thereforenot retain stored contents if deactivated. Examples of volatile memoriesinclude random access memories (RAM), dynamic random access memories(DRAM), static random access memories (SRAM), and other forms ofvolatile memories known in the art. Memory 145, in some examples, alsoinclude one or more computer-readable storage media. Memory 145 may beconfigured to store larger amounts of information than volatile memory.Memory 145 may further be configured for long-term storage ofinformation as non-volatile memory space and retain information afteractivate/off cycles. Examples of non-volatile memories include magnetichard disks, optical discs, floppy disks, Flash memories, or forms ofelectrically programmable memories (EPROM) or electrically erasable andprogrammable (EEPROM) memories. Memory 145 may store programinstructions and/or data associated with one or more of the modulesdescribed in accordance with one or more aspects of this disclosure.

Processing circuitry 143 and memory 142 may provide an operatingenvironment or platform for computation engine 116, which may beimplemented as software, but may in some examples include anycombination of hardware, firmware, and software. Processing circuitry143 may execute instructions and memory 142 may store instructionsand/or data of one or more modules. The combination of processingcircuitry 143 and memory 142 may retrieve, store, and/or execute theinstructions and/or data of one or more applications, modules, orsoftware. Processing circuitry 143 and memory 142 may also be operablycoupled to one or more other software and/or hardware components,including, but not limited to, one or more of the components illustratedin FIG. 2.

Computation engine 116 may perform operations described using software,hardware, firmware, or a mixture of hardware, software, and firmwareresiding in and/or executing at computing device 320. Computation engine116 may execute each of the module(s) with multiple processors ormultiple devices. Computation engine 116 may execute one or more of suchmodules as a virtual machine or container executing on underlyinghardware. One or more of such modules may execute as one or moreservices of an operating system or computing platform. One or more ofsuch modules may execute as one or more executable programs at anapplication layer of a computing platform.

One or more input devices 140 of computing device 106 may generate,receive, or process input. Such input may include input from a keyboard,pointing device, voice responsive system, video camera, biometricdetection/response system, button, sensor, mobile device, control pad,microphone, presence-sensitive screen, network, or any other type ofdevice for detecting input from a human or machine.

One or more output devices 144 may generate, transmit, or processoutput. Examples of output are tactile, audio, visual, and/or videooutput. Output devices 144 may include a display, sound card, videographics adapter card, speaker, presence-sensitive screen, one or moreUSB interfaces, video and/or audio output interfaces, or any other typeof device capable of generating tactile, audio, video, or other output.Output devices 144 may include a display device, which may function asan output device using technologies including liquid crystal displays(LCD), quantum dot display, dot matrix displays, light emitting diode(LED) displays, organic light-emitting diode (OLED) displays, cathoderay tube (CRT) displays, e-ink, or monochrome, color, or any other typeof display capable of generating tactile, audio, and/or visual output.In some examples, computing device 106 may include a presence-sensitivedisplay that may serve as a user interface device that operates both asone or more input devices 140 and one or more output devices 144.

One or more communication units 142 of computing device 106 maycommunicate with devices external to computing device 106 (or amongseparate computing devices of a distributed computing system forexecuting a distributed form of image processor 118) by transmittingand/or receiving data, and may operate, in some respects, as both aninput device and an output device. In some examples, communication units142 may communicate with other devices over a network. In otherexamples, communication units 142 may send and/or receive radio signalson a radio network such as a cellular radio network. Examples ofcommunication units 142 include a network interface card (e.g. such asan Ethernet card), an optical transceiver, a radio frequencytransceiver, a GPS receiver, or any other type of device that can sendand/or receive information. Other examples of communication units 142may include Bluetooth®, GPS, 3G, 4G, and Wi-Fi® radios found in mobiledevices as well as Universal Serial Bus (USB) controllers and the like.

Computing device 106 includes image processor 118, which in the exampleincludes one or more executable modules configured to analyze images120, accurately align them, and generate a composite image 122 of theintegrated circuit. While described here with respect to an integratedcircuit, imaging system 100 may be applied to align and stitchhigh-resolution images of tissue, brain cells or other cells, or othersubjects of microscope scanning for which multiple high-resolutionimages at different locations are captured.

For example, image processor 118 may include tile generator 126(detailed further in FIG. 3, below). Tile generator 126 may select pairsof consecutive or proximate images from the set of images 120. Tilegenerator 126 may select pairs of images to process based on, forexample, the order in which the images were captured, or in otherexamples, based on a corresponding set of microscope stage coordinatesindicating the approximate position of microscope stage 110 (FIG. 1)when the images were captured and further indicating overlap of the pairof images. For each image in the pair of images, tile generator 126 maydetermine an approximate overlap region indicating a section of thefirst image that depicts a same area of the integrated circuit as indepicted in the overlap region of the second image. Tile generator 126may then, for each image of the pair, partition the correspondingoverlap region into a grid of tiles. For example, the overlap region mayinclude a 3×3 grid of tiles, or a differently dimensioned grid of tiles(e.g., 3×4, 4×4, 100×100).

Image processor 118 may include feature extractor 128. Feature extractor128 may identify one or more distinctive or distinguishable featuresfrom the overlap region in each image of the pair of images. In order toincrease the accuracy of the image alignment by more evenly distributingthe identified feature points, feature extractor 128 may identify atleast one feature from each tile of the grid, as previously partitionedby tile generator 126, using a feature recognition algorithm. If featureextractor 128 is unable to identify at least one feature from every tilein an image or is unable to identify a sufficient number of featuresoverall in the overlap region, tile generator 126 may expand the overlapregions of the pair of images, allowing for a larger number of pixelsfrom which to search for feature points. Feature extractor 128 may thenagain attempt to identify the features from the expanded overlapregions. Alternatively or additionally, tile generator 126 may reducethe number of tiles in the grid to reduce the corresponding number offeature points that feature extractor 128 is to identify.

In some cases, feature extractor 128 first enhances the contrast andbrightness in the overlap regions. These local enhancements may bringout the delayering noise as features in the non-circuit areas to improveimage registration. For example, polishing marks from mechanical polishmay be useful for alignment purposes. An original circuit image may bevisually dark. A simple histogram equalization may not be able toenhance all of the edge information in the image. Instead, using acontrast normalization step for instance, feature extractor 128 canenhance all high frequency content in the image so as to provide strongfeature points.

As noted above, using sampled feature points that are distributed acrossthe overlap regions may improve alignment because (1) repeated patternsin circuit images can fool matching points, and (2) noisy patches on keydominant can throw off the ranking of features. Feature matcher 130 mayuse more highly-ranked features from each of the tiles to performfeature matching and alignment. Moreover, if there are repetitivecircuit structures, then it can be difficult to disambiguate among thefeature points. Also, if there are noisy patches (dust or artifactsintroduced by chip delayering artifacts, electron discharge, then thelist of feature points is skewed when considering the list as a whole).By sampling the feature points across tiles, feature extractor 128enforces selectively sampling across the overlap regions such that thereis sufficient distribution of unique feature points for comparison. Thenumber of feature points per tile region can be set as a threshold basedon noise level of the imagery or when the tile has few textures. Thatis, more points may be needed when the tiles have few features or arenoisy.

Image processor 118 may include feature matcher 130. Once featureextractor 128 has identified a distributed set of distinguishing featurepoints in each tile of the overlap region in each of the pair of images,feature matcher 130 may attempt to match identified feature points inthe first image with corresponding identified feature points in thesecond image. If feature matcher 130 is able to match a configurablethreshold number of feature points between the two images, featurematcher 130 may align the two images relative to one another and recordtheir alignment information as a pairwise registration indicating atleast one transformation matrix required to translate, scale, or rotateone of the images to approximately match the overlap region of thatimage with the other image in the pair. The one or more transformationmatrices may include at least one of a translation vector, a rotationmatrix, or scale matrix. In some examples, feature matcher 130 mayassign each pairwise registration a “confidence rating” indicating thenumber and/or relative distinctiveness of successfully matched featurepoints between the pair of images. For example, highly distinctivefeatures identified in each overlap region of a pair of images andmatched with one another increases the confidence rating of the pairwiseregistration and corresponding transformation matrices. As described infurther detail below, bundle adjuster 132 may use the confidence ratingsfor pairwise registrations as weights in a bundle adjustment operationfor adjusting the transformation matrices of multiple images.

Image processor 118 may include bundle adjuster 132 (detailed further inFIG. 4, below), in part to address seams in the composite image(s) to begenerated. In some cases, one pairwise registration may initiallyconflict with one or more other pairwise registrations. For example, apairwise registration between a first and a second image, and a pairwiseregistration between the second and a third image, may conflict with apairwise registration between the third image and the first image.Bundle adjuster 132 may attempt to resolve such discrepancies (reducingthe net error) by adjusting one or more of the pairwise registrations byan amount based on, for example, the number of discrepanciescorresponding to a particular image or pair of images, the extent ordegree of the alignment discrepancies, and/or the previously assignedconfidence ratings for each pairwise registration.

Bundle adjuster 132 may apply one or more constraints. Constraints mayinclude motion constraints and distance constraints, where “motion”refers to the displacement between two overlaps in translation, scale,and rotation. For example, bundle adjuster 132 may adjust the alignmentsby in part requiring that no translation vector for an image can exceeda maximum translation distance from the initial microscope stagecoordinates of the image. In the event that bundle adjuster 132determines that an adjustment for one or more images exceeds such athreshold, bundle adjuster may either assign that translation vector alow confidence rating or exclude that translation vector from the bundleadjustment altogether. Once bundle adjuster 132 has “averaged out” thediscrepancies between the individual pairwise registrations, bundleadjuster 132 may stitch images 120 into composite image 122 based on theadjusted pairwise registrations, now indicative of “global” coordinatesfor the images. Computing device 106 may then output composite image 122via output device(s) 144. In some examples, due to repetitive patternswithin the circuits, ambiguous alignment along the pattern may result inbundle adjuster 132 generating more than one composite image 122 fromthe images 120. The motion and distance constraints may cause bundleadjuster 132 to exclude from outputting the one or more composite images122 in which one or more translation vectors exceeds a maximum distancefrom the microscope stage coordinates, or alternatively, only outputsthe composite image 122 in which translation vectors are collectivelyminimized (i.e., most consistent with the original microscope stagecoordinates).

Image processor 118 may include one or more additional modules tofurther increase the accuracy and/or speed of the image-alignmentprocess. For example, image processor 118 may include frequency analyzer134 (detailed further in FIG. 5, below). Frequency analyzer 134 maycompute a Fast Fourier Transform (FFT) for the images 120 to determineperiodic features, such as repetitive patterns within the images.Frequency analyzer 134 may compare the frequency data to the results ofbundle adjuster 132 to further refine the image alignment. For example,frequency analyzer 134 may exclude a determined image alignment (orprospective composite image) that does not match the frequency data.

Image processor 118 may also include a machine-learning system, such asdeep neural network (DNN) 136 (detailed further in FIG. 6, below). DNN136 may represent software executable by computation engine 116, or acombination of hardware and software. DNN 136 may receive a set oftraining data, learn a set of circuit structures or elements from thetraining data set, and align images 120 to match the learned circuitelements. In some examples, the training data set may include a libraryof images of integrated circuits. In other examples, the training dataset may include frequency data indicating common patterns and structuresappearing in the same figures to be aligned.

In some cases, feature matching for pairwise alignments may be unable tosatisfy the threshold number of required features to satisfy thecriteria. For example, a pair of images may have few feature points,small overlapping regions, or a large estimated motion. SEM 104 mayinclude multiple ICDs 108 configured to simultaneously capture multipleimages. Such images may be referred to as “subfields.” The geometricalignment of these multiple images can be calibrated so that systematicmisalignment can be estimated. Although the relative positions of theseimages drift as time goes by, the change is relatively small. Imageprocessor 118 may use the systematic misalignment parameters if theerror from the real-time alignment is larger than the drift error.

In order to stitch large regions, relative position between the SEM 104and integrated circuit 102 may need to change. In some examples, SEM 104is kept still and integrated circuit 102 on stage 110 moves. There willbe a systematic error related to the direction of the stage 110 motionto the alignment between two microscope images. If motions between twomicroscope captures are poor, image processor 118 may replace thealignment from some prior information (e.g., an average motion betweenall neighbor alignments in the same stage moving direction, etc.). Imageprocessor 118 can use system calibrations of SEM 104 in order to dosmall-scale alignments for images for SEM 104, e.g., to account forstage 110 change errors, image process 118 may use previously-alignedimages as a pattern and use that pattern to replace the current failedalignments. The systematic misalignment due to stage errors isrelatively larger but is useful. In this way, image processor 118 maystill facilitate image alignment even where few feature points areavailable for pairwise matching, for instance.

FIG. 3 is a flowchart illustrating an example mode of operation for animage processor for processing images of an integrated circuit,according to techniques of this disclosure. The mode of operations 300depicted in FIG. 3 includes three categories of image-processingalgorithms. Feature-based image-alignment 302, such as may includedistributed feature-matching and bundle adjusting that provides directmapping from pixel to polygon. Feature-based matching requires tuningand calibration at each stage of processing, and therefore can beexpensive or resource-intensive. However, some stages of feature-basedalgorithms may be accelerated on a graphics processing unit (GPU).

Frequency-based image alignment 304, such as extracting image frequencyinformation with a Fast Fourier Transform, provides global imageanalysis that displays a high tolerance to local image noise. However,frequency-based image processing requires a relatively high number ofsimilar cells to provide accurate clustering data. FFT computations maybe optimized on GPUs. Core-poly based analysis 308 of the images mayinclude FFT segmentation support.

As described herein, robust hybrid image analysis 310 combinespixel-based information with statistical-based information foralignment. This combines both the local and global analyses. As shown inFIG. 3, pixel-based information is gathered from feature-based imagealignment 302. The phrase “pixel-based” refers to the fact that theanalysis does not rely on an understanding of the structure itself. Sometechniques of this disclosure further include structural-basedinformation in combination with a frequency-based alignment algorithmcomponent. Here, an image processor 118 may align and stitch imagesbased on the matching frequency content of the images, i.e., thefrequency information has codified information about the dominantstructure beyond what is available in the feature-based approach.

A deep neural network (DNN) alignment component 306 gathers informationabout the actual circuit itself. DNN 306 aligns images based on objectparts, e.g., integrated circuit structures and elements (e.g.interconnects, transistors, etc.). This DNN component 306 has thehighest level of semantic information (e.g. actually encodes informationabout the circuit cells) to help in alignment. Machine-learning, such aswith a DNN, is fully data-driven, and does not rely on repetitive gridpatterns to produce accurate results. However, in order to functionproperly, a DNN requires an initial training data set, and may requirecalibration to confirm that the DNN has drawn correct inferences fromthe training data. Deep learning acceleration is already optimized onGPUs.

Alignment process 300 may further include integrated tuning andoptimization 312, which may incorporate a larger training set forfine-tuning.

FIG. 4 is a flowchart depicting an example image-processing pipeline 400for processing images of an integrated circuit, according to techniquesof this disclosure. The example method depicted in FIG. 4 may beperformed computing device 106 (FIG. 1). In the example method depictedin FIG. 4, processing images of an integrated circuit includesfeature-based image alignment (410). Feature-based image alignment mayinclude distributed feature-point matching, detailed further in FIG. 5,as well as a bundle adjustment to reduce a net error associated with thealignment of the results of the feature-point matching.

Image-processing pipeline 400 may include structural analysis (412).Structural analysis may include, for example, extracting frequency datadescribing the frequency of repeated structures and patterns within theimages of the integrated circuit. Consecutive images may be aligned inorder to increase or maximize the consistency with frequency data forthe rest of the image.

Image-processing pipeline 400 may include structure extraction (414).Structure extraction may include the use of machine-learning, such as aDNN, to identify expected structures within the chip images and alignthe images to produce those structures.

Finally, image-processing pipeline 400 may include outputting acomposite image of an integrated circuit, displaying the internalcircuit structure as a plurality of two-dimensional polygons (416). Eachpolygon may correspond to a different type of circuit structure, such asadder, multiplier, etc.

FIG. 5 is a block diagram illustrating distributed feature-matching of apair of images of an integrated circuit, as may be performed by tilegenerator 126, feature extractor 128, and/or feature matcher 130 ofimage processor 118 of FIG. 2, according to techniques of thisdisclosure. FIG. 5 depicts a pair of images 146, 148 selected fromimages 120 (FIG. 1). Image processor 118 may determine that image 146has an overlapping region 150 with image 148, based on the images'microscope stage coordinates, indicating the position of the microscopestage with respect to ICD 108 (FIG. 1) when ICD 108 captured the images.Image processor 118 may align image 146 with image 148 by matchingindividual feature points 154 within overlapping region 150. Accordingto some aspects of this disclosure, image processor 118 may increase theaccuracy of the image alignment by using sampled feature points 154 thatare distributed across the overlap regions 150, by partitioning overlapregion 150 into a plurality or grid of tiles 152, and searching for atleast one feature point 154 from every tile 152. Image processor 118samples feature points across distributed tiles 152 because repeatedpatterns in integrated circuit images can “fool” matching points, and“noisy” patches on key dominant can throw off “ranking” (i.e., aconfidence rating based on “distinctiveness”) of feature points 154. Ina traditional image-alignment algorithm, image processor 118 may use thestrongest (i.e., most distinctive) feature points by first sorting theentire set of feature points 154 in the overlap regions 150. However, ifthere are repetitive circuit structures within the images, then it maybe difficult for image processor 118 to disambiguate among the featurepoints 154. Also, if there are noisy patches (e.g., dust or otherartifacts introduced by chip delayering, or electron discharge) then thelist of feature points 154 may be skewed when considering the list as awhole.

Instead, by sampling the feature points 154 across tiles 152, imageprocessor 118 enforces the need to selectively sample across the overlapregions such that there is sufficient distribution of unique featurepoints 154 for comparison. Image processor 118 (or a user) may determinethe number of feature points 154 per tile region 152 as a thresholdbased on noise level of the imagery—i.e., image processor 118 mayrequire more feature points for alignment when the images are noisy.

For each pair of overlapping images 146 and 148, image processor 118 mayenhance image data to increase feature point detections (e.g., contrastenhancement, detailed in FIG. 8, below), detect feature points 154evenly across overlap region 150 using tile-normalization, computefeature descriptors for each feature point, match corresponding featurepairs between images 146 and 148, apply symmetric match to reduce falsematches, and apply motion and distance constraints to remove outliers.

FIG. 6 is a flowchart illustrating an example mode of operation ofdistributed-feature-matching a pair of images “A” and “B” of anintegrated circuit, according to techniques of this disclosure. First,an image processor computes an estimated overlap region for the twoimages, based on a set of image coordinates, such as a set of microscopestage coordinates (610). The image processor may partition the overlapregion for each image into a plurality of normalized tiles, and thenanalyze each tile for one or more distinctive feature points (604, 614).Then, the image processor may compute a set of feature descriptorsindicative of the identified feature points (606, 616). The imageprocessor may perform symmetric matching between the identified featurepoints in Image A and the identified feature points in Image B(608)—i.e., the algorithm is run from A to B as well as from B to A, inorder to reduce the number of false matches. Image processor may applyone or more motion constraints (610), limiting the displacement betweentwo overlapping regions during alignment, as well as one or moredistance constraints (612), limiting the displacement of any oneparticular image during alignment. The example operation depicted inFIG. 6 may output a set of global image coordinates comprising inliers(i.e., having excluded the “outlier” coordinates) and a description ofthe motion or displacement of each circuit image with respect to itsinitial microscope stage coordinates.

FIG. 7 depicts a bundle adjustment of pairwise registrations of imagesof an integrated circuit, as may be performed by bundle adjuster 132 ofFIG. 2, according to techniques of this disclosure. Aligning andstitching images through feature-matching alone may result in acomposite image with a number of visible “seams” between the individualimages. These alignment discrepancies may be due in part to therepetitious nature of the circuits' internal structure. To addressvisible seams in the composite mosaic image, bundle adjuster 132 ofimage processor 118 (FIG. 2) may perform a bundle adjustment.

Bundle adjuster 132 finely tunes the registration parameters acrossmultiple images as a bundle to refine the final registration result.Because most of the images have more than one overlapping neighbor, andnot all image featured-based-matching registrations are correct, bundleadjustment may thus be used to improve, and in some cases optimize, theimage stitching. In general, bundle adjustment refers to simultaneouslyrefining the alignment results for each image in a bundle. The alignmentcoordinate refers to the 3D vector that provides the best alignment forthe overlap region, and the net result of bundle adjustment is thebest-fit adjustment based on minimizing the net error associated withthe alignment of the images. Bundle adjuster 132 outputs the new globalcoordinates of all images.

In some examples, to determine the global coordinates of each image,bundle adjuster 132 solves the following optimization problem:

$\underset{\overset{\rightarrow}{x}}{\arg\;\min}\left\lbrack {{\sum\limits_{a,{b \in {overlap}}}{w_{ab}\left( {{R_{ab}{\overset{\rightarrow}{x}}_{a}} - {\overset{\rightarrow}{x}}_{b} - {\overset{\rightarrow}{t}}_{ab}} \right)}^{2}} + {\beta{\sum\limits_{i}\left( {{\overset{\rightarrow}{x}}_{i} - {\overset{\rightarrow}{s}}_{i}} \right)^{2}}}} \right\rbrack$where {right arrow over (x)}_(i) is the coordinate of image i, andR_(ab) and {right arrow over (t)}_(ab) are the rotation matrix andtranslation vector from image a to image b. {right arrow over (s)}_(i)is the reference coordinate (or fiducial point) for image i. Thesefiducial points are predetermined points as a reference to the circuits.β is the regularization parameter, which controls how much influence{right arrow over (s)}_(i) has. w_(ab) is the weight for the motionestimate between image a and b. By weighting the motion estimates, thebundle adjuster 132 performs a weighted bundle adjustment that biasesadjustments indicated by better pair matches. The weight may correspondto the confidence rating determine by feature matcher 130. For example,in some cases, the number of matches in a pairwise registrationconstitutes the weight for the translation vector of the pairwiseregistration. In general, bundle adjuster 132 adjusts the transformationwith the greatest weight by the least amount.

This bundle adjustment may consist of an iterative refinement of therelative motion parameters for a group of images. The first bundleadjustment may include the refinement of the relative motions among aset of subfield images. The second bundle adjustment may be applied to aregion which includes hundreds and thousands of microscope images. Therefinement can be extended to a physical layer or the entire layer ofmany physical layers. FIG. 7 depicts an example of the results of bundleadjuster 132. Dashed circles represent the initial microscope stagecoordinates for each image. Solid circles represent the finalbundle-adjusted coordinates for each image within the final compositeimage. Reference point 710 is the fiducial point at the microscope imagecoordinates stay unchanged.

FIG. 8 depicts a frequency analysis of images of an integrated circuit,as may be performed by frequency analyzer 134 of FIG. 2, according totechniques of this disclosure. In addition to performing distributedfeature-matching and a bundle adjustment, frequency analyzer 134 mayincrease the accuracy of the image alignment by performing a frequencyanalysis on images 120. For example, frequency analyzer 134 may extractstructural-based information from the circuit images by computing a FastFourier Transform (FFT) for images 120. The frequency information hascodified information about the dominant structure beyond what isavailable in the feature-based alignment modules. Frequency analyzer 134may conduct an FFT analysis over the larger image regions instead of theoverlap regions 150 (FIG. 5). Image processor 118 may then align andstitch images in part based on matching frequency content of overlappingimages, or alternatively, eliminate previously determined alignmentsrevealed to have inconsistent frequency content.

The example of FIG. 8 depicts a computed 2-dimensional FFT 812 for image810 (one of images 120 of FIG. 2). A similar FFT 816 computed over asmaller sub-region 814 of image 812 will often result in a similarfrequency signature 818 as FFT 812, because the circuit layout is highlyrepetitive, and circuits are often structured around a grid duringmanufacturing. Image processor 118 may use the frequency information toalign signal peaks (e.g. along the x and y dimensions) based on the gridstructure. In contrast to the pixel-level operations of feature-basedalignment, image processor 118 may perform frequency-based alignment ata more global scale (i.e., at the scale of entire images). Frequencyanalysis provides information about circuit-cell repetition, though notthe actual contents of the cells.

In some examples, frequency analyzer 134 may also perform an initialfrequency analysis to enhance low-contrast circuit images, in which theoriginal circuit image may be too visually dark, obscuring most featurepoints and inhibiting feature extraction. A simple histogramequalization may not be able to enhance all of the edge information inthe image. Instead, using a contrast normalization step, the imageprocessor 118 can enhance the contrast of all high-frequency content inthe image so as to provide strong feature points for stitching.

FIG. 9 is a flowchart depicting a method of aligning images of anintegrated circuit using a Deep Neural Network (DNN) 136 of FIG. 2,according to techniques of this disclosure. Image processor 118 mayimplement a machine-learning approach to “learn” about individualcircuit structures, and use the learned information for image alignment.For example, image processor 118 may implement a DNN 136 that is trainedusing data sets from circuit-cell image libraries, available fromsemiconductor fabrication companies, for instance (910). These celllibraries have identified patterns and shapes corresponding to eachdifferent circuit function (e.g. AND, NAND, NOR, XOR). Once trained, DNN136 can correctly identify the circuit pattern associated with eachcell. Any seams created during image stitching may be resolved bypattern-matching against the outputs of DNN 136. For example, DNN 136may specify the expected shape (912), and image processor 118 may thenadjust or refine the translation, rotation, and/or scaling parameters ofthe image being stitched to match the expected shape (914). DNN 136 mayalign images based on circuit sub-components, such as, interconnects,transistors, etc. Accordingly, DNN 136 has the highest level of semanticinformation in that it actually encodes information about the circuitcells to aid in alignment.

FIG. 10 is a flowchart illustrating a method of aligning images of anintegrated circuit using frequency analysis to generate a set oftraining data for a DNN, according to techniques of this disclosure. Insome examples, rather than using a circuit image library as a trainingset, DNN 136 may instead use frequently-occurring circuit-cellstructures within the same integrated circuit images 120 that arecurrently being aligned. For example, DNN 136 may work in conjunctionwith a frequency-analysis module to determine common patterns of cellswithin the circuit, and then align images 120 to match these frequentcell patterns. In one example, image processor 118 (FIG. 2) may useeither Local Binary Pattern (LDP 1012A) (local gradient based onintensity) or Descriptors for Shape and Letters (DSL 1012B) (pattern-and location-specific), as a vector description of any grid region.Then, image processor 118 may implement a clustering method to identifycommon cell patterns. Finally, using these common cell patterns, imageprocessor 118 may align images based on the expected shapes and/orpatterns of circuits.

In some examples, DNN 136 may also work in conjunction with a frequencyanalysis 134 to resolve imaging noise, relying on the repetitious natureof circuit structures. Once image processor 118 has identified cellstructures (as described above), image processor 118 may then determinea list of reference cells (e.g., the most representative cells, in thecenter of the cluster of similar cells). Image processor 118 may thendesignate any outlier cells (e.g. ones with a noisy patch) as anomaliesand correct them using the reference cells as a guide. Any cell that hasa non-aligned seam can also be treated as an anomaly, and thus, imageprocessor 118 can then adjust/refine the motion parameters of the imagebeing stitched to match the expected shape from the reference cells.

FIG. 11 is a flow chart depicting a method of processing images of anintegrated circuit, according to techniques of this disclosure. An imageprocessor, such as a set of executable instructions within a computingdevice, receives two or more images depicting overlapping sections of anintegrated circuit. The image processor, based on a respective set ofmicroscope stage coordinates associated with each image, determines anapproximate overlapping region in each image 1110).

The image processor may partition each overlap region into a pluralityof tiles from which to search for distinctive feature points (1112). Inorder to improve the accuracy of alignment, the image processor may beconfigured to identify at least one feature point from every tile in therespective overlap region (1114). Then, for each pair of overlappingimages, the image processor attempts to match feature points from thefirst overlapping region on the first image with feature points from thesecond overlapping region on the second image, and align the two imagesso that matched feature points are precisely aligned (1116). Thealignment, based on matched feature points, results in a set of pairwiseregistrations comprising a set of translation vectors, indicating amagnitude and a direction by which to translate each image, from itsinitial stage coordinates, to the aligned position (1118). In someexamples, the set of translation vectors may also be assigned aconfidence rating based on the strength and/or number of matched featurepoints generating that respective vector. To reduce a net error, orinconsistencies, among two or more pairs of translation vectors, imageprocessor may perform a bundle adjustment on the pairs of overlappingimages (1120). The bundle adjustment may be weighted by confidenceratings for pairwise images, e.g., based on metadata from thefeature-matching indicating the number of matches, the effect of one ormore constraints, and so forth. For example, image processor 118 mayadjust or refine the translation vector for one or more images based onthe number or degree of inconsistencies associated with the respectiveimage, and/or based on the determined confidence ratings associated withthe image. The bundle adjustment results in a set of global imagecoordinates, from which the image processor may stitch the individualimages into a single composite image of the circuit.

The techniques of this disclosure may be amenable to applications beyondtrusted hardware. For example, electron microscopes have been used totake images of body tissue and in particular brain cells. Theapplication there include the study of the neurons and synapticconnections. The need is similar in that the images are low contrast,have highly repetitive structures, and are composed of many stitchedseams. The proposed algorithm can be used in those applications as well.

In one or more examples, the functions described may be implemented inhardware, software, firmware, or any combination thereof. If implementedin software, the functions may be stored on or transmitted over, as oneor more instructions or code, a computer-readable medium and executed bya hardware-based processing unit. Computer-readable media may includecomputer-readable storage media, which corresponds to a tangible mediumsuch as data storage media, or communication media, which includes anymedium that facilitates transfer of a computer program from one place toanother, e.g., per a communication protocol. In this manner,computer-readable media generally may correspond to (1) tangiblecomputer-readable storage media, which is non-transitory or (2) acommunication medium such as a signal or carrier wave. Data storagemedia may be any available media that can be accessed by one or morecomputers or one or more processors to retrieve instructions, codeand/or data structures for implementation of the techniques described inthis disclosure. A computer program product may include acomputer-readable storage medium.

By way of example, and not limitation, such computer-readable storagemedia can comprise RAM, ROM, EEPROM, CD-ROM or other optical diskstorage, magnetic disk storage, or other magnetic storage devices, flashmemory, or any other medium that can be used to store desired programcode in the form of instructions or data structures and that can beaccessed by a computer. Also, any connection is properly termed acomputer-readable medium. For example, if instructions are transmittedfrom a website, server, or other remote source using a coaxial cable,fiber optic cable, twisted pair, digital subscriber line (DSL), orwireless technologies such as infrared, radio, and microwave, then thecoaxial cable, fiber optic cable, twisted pair, DSL, or wirelesstechnologies such as infrared, radio, and microwave are included in thedefinition of medium. It should be understood, however, thatcomputer-readable storage media and data storage media do not includeconnections, carrier waves, signals, or other transient media, but areinstead directed to non-transient, tangible storage media. Disk anddisc, as used herein, includes compact disc (CD), laser disc, opticaldisc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wheredisks usually reproduce data magnetically, while discs reproduce dataoptically with lasers. Combinations of the above should also be includedwithin the scope of computer-readable media.

Instructions may be executed by one or more processors, such as one ormore digital signal processors (DSPs), general purpose microprocessors,application specific integrated circuits (ASICs), field programmablelogic arrays (FPGAs), or other equivalent processing, integrated and/ordiscrete logic circuitry. Accordingly, the term “processor,” as usedherein may refer to any of the foregoing structure or any otherstructure suitable for implementation of the techniques describedherein. In addition, in some aspects, the functionality described hereinmay be provided within dedicated hardware and/or software modules. Also,the techniques could be fully implemented in one or more circuits orlogic elements.

The techniques of this disclosure may be implemented in a wide varietyof devices or apparatuses, including a wireless handset, an integratedcircuit (IC) or a set of ICs (e.g., a chip set). Various components,modules, or units are described in this disclosure to emphasizefunctional aspects of devices configured to perform the disclosedtechniques, but do not necessarily require realization by differenthardware units. Rather, as described above, various units may becombined in a hardware unit or provided by a collection ofinteroperative hardware units, including one or more processors asdescribed above, in conjunction with suitable software and/or firmware.

What is claimed is:
 1. A method of stitching images to generate acomposite image of an object, comprising: by a computing system, foreach pair of two or more pairs of the images: partitioning a firstoverlap region of a first image of the pair into a first plurality ofgrid tiles and partitioning a second overlap region of a second image ofthe pair into a second plurality of grid tiles, the first overlap regionand the second overlap region being overlapping regions of the firstimage and the second image; extracting at least one feature point fromeach of the first plurality of grid tiles and at least one feature pointfrom each of the second plurality of grid tiles, wherein the featurepoint defines a feature on an image, the feature being on a submicron ornanoscale level; determining corresponding feature points between the atleast one feature point of the first overlap region and the at least onefeature point of the second overlap region; and aligning the first imagewith the second image based on the corresponding feature points todetermine at least one transformation matrix for the pair fortransforming the first overlap region of the first image to the secondoverlap region of the second image; computing, by the computing systembased on the respective at least one transformation matrices for the twoor more pairs, global coordinates for each of the images by performing abundle adjustment for at least two of the two or more pairs of theimages to reduce a net error associated with the alignment of theimages; stitching, by the computing system, the images based on therespective global coordinates for the images to generate the compositeimage; and outputting, by the computing system, at least one of thecomposite image and the respective global coordinates for the images. 2.The method of claim 1, further comprising: determining, by the computingsystem for a first pair of the two or more pairs of the images, thatless than a threshold number of feature points are in at least one ofthe first plurality of grid tiles for the first image of the first pairor the second plurality of grid tiles for the second image of the firstpair; and expanding, by the computing system, the first overlap regioninto an expanded first overlap region and the second overlap region intoa second expanded overlap region, wherein partitioning the first overlapregion and the second overlap region comprises partitioning the expandedfirst overlap region and the expanded second overlap region.
 3. Themethod of claim 1, further comprising: determining, for each pair of thetwo or more pairs of images, a confidence rating based at least in parton the corresponding feature points, wherein performing the bundleadjustment comprises computing the global coordinates based at least inpart on the confidence rating.
 4. The method of claim 1, whereinperforming the bundle adjustment comprises applying a neural networkhaving a model trained to identify images of integrated circuit elementsto align misaligned portions of an integrated circuit elementdistributed among two or more images.
 5. The method of claim 1, furthercomprising: computing, by the computing system, a systematic error inimage alignment caused by stage motion by a stage supporting the object,wherein computing the global coordinates for each of the imagescomprises computing, by the computing system based at least in part ofthe systematic error in image alignment, the global coordinates for eachof the images.
 6. The method of claim 1, wherein performing the bundleadjustment comprises excluding a pair of images from the bundleadjustment when a motion specified by the at least one transformationmatrix for the pair of images exceeds a motion constraint.
 7. The methodof claim 6, further comprising: determining, by the computing system,for one of the at least two distinct composite images, whether a motionspecified by the at least one transformation matrix for a first pair ofthe images is larger than a threshold; and excluding the first pair ofimages from the bundle adjustment as an outlier in response todetermining the motion specified by the at least one transformationmatrix for the first pair of the images is larger than the threshold. 8.The method of claim 7, wherein the threshold is based at least in parton a set of microscope stage coordinates.
 9. The method of claim 1,further comprising: computing, by the computing system, a Fast FourierTransform (FFT) to determine global frequency data for the compositeimage, wherein outputting the composite image comprises outputting thecomposite image only if the global frequency data indicates integratedcircuit elements of the composite image are globally consistent.
 10. Themethod of claim 1, wherein each of the at least one transformationmatrices is one of a translation matrix, a rotation matrix, or a scalematrix.
 11. An integrated circuit imaging system comprising: animage-capture device configured to capture images of an integratedcircuit; and a computation engine executing on one or more processorsand configured to, for each pair of two or more pairs of the images:partition a first overlap region of a first image of the pair into afirst plurality of grid tiles and partition a second overlap region of asecond image of the pair into a second plurality of grid tiles, thefirst overlap region and the second overlap region being overlappingregions of the first image and the second image; extract at least onefeature point from each of the first plurality of grid tiles and atleast one feature point from each of the second plurality of grid tiles,wherein the feature point defines a feature on an image, the featurebeing on a submicron or nanoscale level; determine corresponding featurepoints between the at least one feature point of the first overlapregion and the at least one feature point of the second overlap region;and align the first image with the second image based on thecorresponding feature points to determine at least one transformationmatrix for the pair for transforming the first overlap region of thefirst image to the second overlap region of the second image, whereinthe computation engine is further configured to compute, based on the atleast one transformation matrices for the two or more pairs, globalcoordinates for each of the images by performing a bundle adjustment forat least two of the two or more pairs of the images to reduce a neterror associated with the alignment of the images, wherein thecomputation engine is further configured to stitch the images based onthe respective global coordinates for the images to generate a compositeimage, and wherein the computation engine is further configured tooutput at least one of the composite image and the respective globalcoordinates for the images.
 12. The system of claim 11, wherein thecomputation engine is further configured to: determine, for a first pairof the two or more pairs of the images, that less than a thresholdnumber of feature points are in at least one of the first plurality ofgrid tiles for the first image of the first pair or the second pluralityof grid tiles for the second image of the first pair; and expand thefirst overlap region into an expanded first overlap region and thesecond overlap region into a second expanded overlap region, whereinpartitioning the first overlap region and the second overlap regioncomprises partitioning the expanded first overlap region and theexpanded second overlap region.
 13. The system of claim 11, wherein thecomputation engine is further configured to determine, for each pair ofthe two or more pairs of images, a confidence rating based at least inpart on the corresponding feature points, wherein performing the bundleadjustment comprises computing the global coordinates based at least inpart on the confidence rating.
 14. The system of claim 11, wherein thecomputation engine is configured to perform the bundle adjustment byapplying a neural network having a model trained to identify images ofintegrated circuit elements to align misaligned portions of anintegrated circuit element distributed among two or more images.
 15. Thesystem of claim 11, wherein the computation engine is further configuredto: compute a systematic error in image alignment caused by stage motionby a stage supporting the object, wherein to compute the globalcoordinates for each of the images the computation engine is configuredto compute, based at least in part of the systematic error in imagealignment, the global coordinates for each of the images.
 16. The systemof claim 11, wherein the computation engine is further configured toperforming the bundle adjustment comprises excluding a pair of imagesfrom the bundle adjustment when a motion specified by the at least onetransformation matrix for the pair of images exceeds a motionconstraint.
 17. The system of claim 16, wherein the computation engineis further configured to: determine, for one of the at least twodistinct composite images, whether a motion specified by the at leastone transformation matrix for a first pair of the images is larger thana threshold; and exclude the first pair of images from the bundleadjustment as an outlier in response to determining the motion specifiedby the at least one transformation matrix for the first pair of imagesis larger than the threshold.
 18. The system of claim 17, wherein thethreshold is based at least in part on a set of microscope stagecoordinates.
 19. The system of claim 16, wherein the computation engineis further configured to: compute a Fast Fourier Transform (FFT) todetermine global frequency data for the composite image, whereinoutputting the composite image comprises outputting the composite imageonly if the global frequency data indicates integrated circuit elementsof the composite image are globally consistent.
 20. A non-transitorycomputer-readable medium comprising instructions that, when executed,cause one or more processors to execute a computation engine configuredto: for each pair of two or more pairs of images of an integratedcircuit: partition a first overlap region of a first image of the pairinto a first plurality of grid tiles and partition a second overlapregion of a second image of the pair into a second plurality of gridtiles, the first overlap region and the second overlap region beingoverlapping regions of the first image and the second image; extract atleast one feature point from each of the first plurality of grid tilesand at least one feature point from each of the second plurality of gridtiles, wherein the feature point defines a feature on an image, thefeature being on a submicron or nanoscale level; determine correspondingfeature points between the at least one feature point of the firstoverlap region and the at least one feature point of the second overlapregion; and align the first image with the second image based on thecorresponding feature points to determine at least one transformationmatrix for the pair for transforming the first overlap region of thefirst image to the second overlap region of the second image; whereinthe instructions further cause the one or more processors to execute thecomputation engine to: compute, based on the respective at least onetransformation matrices for the two or more pairs, global coordinatesfor each of the images by performing a bundle adjustment for at leasttwo of the two or more pairs of the images to reduce a net errorassociated with the alignment of the images; stitch the images based onthe respective global coordinates for the images to generate thecomposite image; and output at least one of the composite image and therespective global coordinates for the images.